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f~^ ■ Abstract 

(N 

5_^ ■ In this paper, the concept of the classical /-divergence (for a pair of measures) is extended to the mixed /- 

divergence (for multiple pairs of measures). The mixed /-divergence provides a way to measure the difference 
between multiple pairs of (probability) measures. Properties for the mixed /-divergence are established, such as 
permutation invariance and symmetry in distributions. An Alexandrov-Fenchel type inequality and an isoperimetric 
type inequality for the mixed /-divergence will be proved and applications in the theory of convex bodies are given. 
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Index Terms 

Alexandrov-Fenchel inequality, f-dissimilarity, /-divergence, isoperimetric inequality. 

I. Introduction 



In applications such as pattern matching, image analysis, statistical learning, and information theory, one often 
needs to compare two (probability) measures and to know whether they are similar to each other Hence, finding 
the "right" quantity to measure the difference between two (probability) measures P and Q is central. Traditionally, 
people use classical Lp distances between P and Q, such as the variational distance and/or the L2 distance. However, 
the family of /-divergences is often more suitable to fulfil the goal than the classical Lp distance of measures. 
k> ' The /-divergence Df{P, Q) of two probability measures P and Q was first introduced in 19], and independently 

^ ; in El, Ell as 

Df{P,Q)= l^f(^)qdfi. (I.l) 

Here, p and q are density functions of P and Q with respect to a measure ij. on X. The idea behind the /-divergence 
is to replace, for instance, the function f{t) = |t — 1| in the variational distance by a general convex function /. 
Hence the /-divergence includes various widely used divergences as special cases, such as, the variational distance, 
the KuUback-Leibler divergence lfT6l . the Bhattcharyya distance ID and many more. Consequently, the /-divergence 
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receives considerable attention (e.g., ||3], lH), lfT4l . ifTSl . 1291 ). We also refer to, for instance H, for more references 
related to the /-divergence. 

Extension of the /-divergence from two (probability) measures to multiple (probability) measures is fundamental 
in many applications, such as statistical hypothesis test and classification, and much research has been devoted to 
that, for instance in ll26l . IZTl . BOl . Such extensions include, e.g., the Matusita's affinity 1241 . 125JI . the Toussaint's 
affinity l35l . the information radius ll34l and the average divergence ll33l . 

The f-dissimilarity Df (Pi, • • • , Pi) for (probability) measures Pi, • • • , P;, introduced in ifTTl . |[T2l for a convex 
function f : M' — > K, is a natural generaUzation of the /-divergence. It is defined as 

Df{Pu---,Pi)= I {{pi,--- ,pi)dfi, 
Jx 

where the p/s are density functions of the P^'s that are absolutely continuous with respect to /i. For a convex 
function /, the function {{x,y) ~ vf{-) is also convex on x,y > 0, and Df{P,Q) is equal to the classical 

/-divergence defined in formula (II. lb . Note that the Matuista's affinity is related to 

I 



x^ . 



and the Toussaint's affinity is related to 

I I 

f{xi,- ■ ■ ,xi) = -Y\_x°;\ Oi > with }a.i = l. 

Inspired by the growing fascinating connections between convex geometry and information theory (e.g., ifTSi , 
I2TI - I23I . l30l . Il36l . l37l ). we introduce special f-dissimilarities, namely the mixed /-divergence and the i-th 
mixed /-divergence. These will be done in Section 2 and Section 5 of this paper Also in Section 2, we establish 
some basic properties of the mixed /-divergence. In Section 3, we focus on the f-dissimilarity and the mixed 
/-divergence for multiple convex bodies. In particular, we show that the general mixed affine surface area - a 
fundamental concept in convex geometry - is a special case of the mixed /-divergence. An Alexandrov-Fenchel 
type inequality and an isoperimetric type inequality for the mixed /-divergence are obtained in Section 4. Section 
5 is dedicated to the i-mixed /-divergence and its related isoperimetric type inequalities. 

II. The Mixed /-Divergence. 

Throughout this paper, let (X, [i) be a finite measure space. For 1 < i < n, let P^ = p^/i and Qi = g^/i be 
probability measures on X that are absolutely continuous with respect to the measure /^j. Moreover, we assume that 
for alH = 1, • • • , n, Pi and qi are nonzero almost everywhere w.rt. the measure /i. We use P and Q to denote the 
vectors of probability measures, or, in short, probability vectors, 

P = (Pl,P2,---,P„), Q = (Ql,Q2,---,Qn)- 

We use p and q to denote the vectors of density functions, or density vectors, for P and Q respectively, 

dV ^ . rfQ . . ^ 

-— =p= (pi,p2,--- ,p„), — — = q= (gi,g2,--- ,?«)• 
d[i an 
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We make the convention that • co = 0. 

Denote by M+ = {x e K : a; > 0}. Let / : (0, oo) ^ 
*-adjoint function /* : (0, oo) — > M+ of / is defined by 



be a non-negative convex or concave function. The 



rit) = tfii/t). 

It is obvious that (/*)* = / and that /* is again convex, respectively concave, if / is convex, respectively concave. 
Let fi : (0, oo) — s- M+, I < i < n, he either convex or concave functions. Denote by f = (/i, /2, • ' ' ; fn) the 
vector of functions. We write 

f*-(/r,/2, ■■•,/:) 

to be the ^-adjoint vector for f . 

Now we introduce the mixed f -divergence for (f , P, Q) as follows. 

Definition II.l. Let (X, fi) be a finite measure space. Let P and Q be two probability vectors on X with density 
vectors p and q respectively. The mixed f -divergence -Dp(P, Q) for (f , P, Q) is defined by 



^r(P.Q)^Xn[/.g)» 

Similarly, we define the mixed /-divergence for (f , Q, P) by 



^f(Q,p) 



'.n 



,Af}n 



dp. 



d/i. 



(II.2) 



(IL3) 



A special case is when all distributions Pi and Qi are identical and equal to a probability distribution P. In this 

case, 

i?f(P,Q) = Z?(/,j,,..,/„)((P,P,--- ,P),(P,P,.-- ,P)) 

n 

1=1 

Remark. The mixed /-divergence as defined in Definition lll.ll is closely related to the f-dissimilarity. In fact, taking 

then the f-dissimilarity is equal to the negative of the mixed /-divergence, namely, 

Dt{Pi,Qi;--- ;P„,g„) = -i?f(P,Q), 

if f is convex. In general, the function f could be neither convex nor concave. However, if, for instance, all fi are 
twice differentiable concave functions, then f is a convex function. Indeed, let A G [0, 1], xi,X2,yi,y2 G IR^ such 
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that 2/1 7^ and 2/2 7^ 0. Put x\ = Xxi + (1 — X)x2 and yx = Aj/i + (1 — X)y2- Then we have for all 1 < fc < n, 

\yxj} I \yi yx yi yx 

where the first inequality is from the concavity of ft and the monotone increasing of t*^'"; while the second 
inequality is from the concavity of t'^/". That is, the functions [yfi{x/y)]^^'^ defined on a:, y G M+ are concave 
for all 1 < i,k < n. Therefore, the Hessian of f can be written as a block matrix with all diagonal matrices being 
positive semi-definite and all off-diagonal blocks equal to 0. Consequently, the Hessian of f is positive semi-definite 
and hence f is convex. 

Let TT E Sn denote a permutation on {1,2,-- ,n} and denote 

One immediate result from Definition III. II is the following permutation invariance for Df{P, Q). 

Proposition II.l (Permutation invariance). Let the vectors f , P, Q be as above, and let it E S{n) be a permutation 
on {1, 2, • • • , n}. Then 

When all (/i, P^, Qi) are equal to (/, P, Q), the mixed /-divergence is equal to the classical /-divergence, denoted 
by Df{P, Q), which takes the form 

DfiP.Q) = D^jj,...j){{P,P,--- ,P),{Q,Q,---,Q)) 



/ - qdfi. 

X \qj 

As f*{t) = t/(l/t), one easily obtains a fundamental property for the classical /-divergence Df{P, Q), namely, 

Df{P,Q)^Df,{Q,P), 

for all (/, P, Q). Similar results hold true for the mixed /-divergence. We show this now. 
Let < fc < n. We write D^^{P, Q) for 

k 



^F,.(p>Q)=/ n 



4=1 



h\'f^,. 



xn 

i=k+l 



f* 



Pi 



dfi. 



Clearly, D^JP, Q) = Df{P, Q) and D^/P, Q) = D^, (Q, P), where 

f = (Jl 5 /2 J ■ ' ■ J /«)■ 

Then we have the following result for changing order of distributions. 
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Proposition II.2 (Principle for changing order of distributions). Let f, P,Q be as above. Then, for any < 
k < n, one has 

In particular, 

Df{P,Q)=Df,{Cl,P). 



Proof: Let < k < n. Then, 

fe 

n 



X 



X 



fl — * 


— n 

xll 

i=k+l 


J. / Pi \ 

/i — g< 


ft —]qi 


J- n 

xll 

i=k+l 


/j — Pi 
. \PiJ _ 






^f.JP.Q), 



f,A: 



where the second equality follows from fi [f^ ] qi 



h \p,)- 



A direct consequence of Proposition III. 2] is the following symmetry principle for the mixed /-divergence. 

Proposition II.3 (Symmetry in distributions). Let i,P,Q,be as above. Then, D^iP, Q)+_Dj, (Pj Q) '■^ symmetric 

in P and Q, namely, 

Z?p(P, Q) + D^, (P, Q) = i?f (Q, P) + ^f. (Q, P)- 

Remark. Proposition lII.2l savs that -Df(P, Q) remains the same if one replaces any triple (/i, P^, Qi) by (/*, Qi, Pi). 
It is also easy to see that, for all Q < k,l < n, one has 

Df(P,Q) = Cf_fc(P,Q) = i?f./Q,P) = i?f.(Q:P)- 
Hence, for all < k,l < n, 

D,-,(P- Q) + ^f.,i(P- Q) = ^f (P, Q) + ^f* (P. Q) 

is symmetric in P and Q. 

Hereafter, we only consider the mixed /-divergence D^(P, Q) defined in formula ( III.2l i. Properties for the mixed 
/-divergence Df{Q,P) defined in ( III.3b follow along the same lines. 
Now we list some important mixed /-divergences. 

Examples. 

(i) The total variation is a widely used /-divergence to measure the difference between two probability measures 
P and Q on {X,fi). It is related to function /(<) = |t — 1|. Similarly, we define the mixed total variation by 



Df(P,Q) = y l[\p. - qt\" dfi. 



It measures the difference between two probability vectors P and Q. 
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(ii) For a G R, we denote by a+ = niax{a,0}. We define the mixed relative entropy or mixed Kullback Leibler 
divergence of P and Q by 



i?KL(p,Q)=i?(/,,..,/,)(p,Q)=/ n 



Pi In — 



dfi, 



where f{t) = t In t. When Pi = P = pji and Qi = Q = qji for all i = 1, 2, • ■ • , ri, we get the following (modified) 
relative entropy or Kullback Leibler divergence 



Dkl{P\\Q) ^ I P 
Jx 



Inl^ 
P 



djjL. 



(iii) For the (convex and/or concave) functions fai{t) = t"\ oii £ K for 1 < i < n, we define the mixed Hellinger 
integrals 



•^ X ,_i 



Pi 1^ 



dfi. 



In particular. 



D 



(t°,t°,--- ,t°) 



p,Q)- / np"9r c'M- 



Those integrals are related to the Toussaint's affinity (see Introduction), and can be used to define the mixed a-Renyi 
divergence 



i^a({p,:iiQor=i) = ^^^(l^f[p°"ir'dfi 



-ln[Z?( 



t° ,*",••• .t") 



;p,Q)]. 



The case a^ = ^, for all i = 1, 2, • • • , n, gives the mixed Bhattcharyya coefficient or mixed Bhattcharyya distance 

of(P,Q), 



P n 



This integral is related to the Matuista's affinity (see Introduction). For more information on the corresponding 
/-divergences we refer to e.g. llTSl . 

III. Applications to convex geometry. 

An important application of the mixed /-divergence arises in the theory of convex bodies. A convex body K in 
M" is a convex, compact subset of M" with non-empty interior We write /Co for the set of all convex bodies in 
M" with the origin in the interior We use \K\ to denote the volume of K and \dK\ to denote the surface area of 
dK, the boundary of K. We write i?2 for the Euclidean unit ball in M" and 5"^^ for the unit sphere in M". The 
usual inner product in E" is denoted by (•, •). 

For K E /Co, the polar body K° of K is defined by 

A'° = {j/ e M" : (x, y) < 1, Vx G K}. 
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The support function of K, hx '■ S" 



^+, is hxiu) = ina,Xx£K{x,u). For x £ dK, kk{x) is the (generahzed) 



Gaussian curvature at x. Then, for a convex body K of class of C^, i.e., whose boundary is C^ with strictly 
positive Gauss curvature everywhere, the curvature function fK{u) : 5"^^ — ;> M is defined by fniu) ~ ^ },s , 
where x G dK is such that the outer normal vector to dK at x is u. We refer to ifTO) . lISTI for more details on 
convex bodies. 



A. General mixed affine surface areas. 

Here, we link the mixed /-divergence with general mixed affine surface areas for convex bodies. We let X ~ 5"~^ 
be the unit sphere in R" and /i be the spherical measure a. Let Ki, . . . ,Kn G /Co be convex bodies of class of 
Cl, and C G S''-\ For 1 < i < n, let 



PkAO 



1 



n\Kt\h],^{0 



QkAO 



n \KA ' 



and define probability measures on 5"" ^ by 

PKi=PK,<7 and Qxi^qK^cr. 

Let fi : (0, oo) -^ R+, 1 < i < n, be convex and/or concave functions. Then, we define D^iJ^Pn^ , ■ ■ ■ , Pk„)j {Qki , • 
by 



S"- 



n 

n 

n 



/< ( -^ ) qx. 



da 



s—i-- 



/. 



m\K°\-AfKM. 






n K,, 



da. 



(111.4) 



This expression is closely related to the general mixed L^ (or L^,) affine surface areas introduced in ll39l . The 
companion expression 

Df{iQK„...,QKj,iPK^,--.,PKj) 



Sn-l 



Sn-1 



n 

n 

n 



.fi[ PK, 

PK, 



n+1 



.h 



\K,\\K° 



da 



I ATI 



nh 



Ki 



da. 



(in.5) 



is closely related to the general mixed L!, (or LV) affine surface areas introduced in 11391 . One can easily obtain that 
both formulas (IIII.4I ) and ( IIII.5I ) are affine invariant. For instance, for all linear transform T with the absolute value of 
its determinant equal to 1, one has D-^[{Ptkii- ■ ■ ,Ptk„), {QtKh- ■ ■ ,Qtk„)) equal to D^[{Pk^, . . . ,Pk„), (Qki, 

The "principle for change of order" (Proposition III.2) implies that D^(^{Pki , • ■ • , Pk,^)i [Qki , • • ■ , Qk„)) is 
identical to D^, {{Qk, , • ■ • , Qk„ ), {Pr^ ,...,PkJ)- 



',K, 



.)) 



,Qk^))- 
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When all Ki are centrally symmetric Euclidean balls, i.e., for alH = 1, . . . , n, Ki = riSJ for some r^ > 0, then 
for all ^ 6 5"^^, hKiiC) ~ r-i, and fxiiO = ^?~^- This implies that p^i — QKi = ;rrff^ ^°'" all 1 < * < "■. and 
hence 

n 
D^Pr.B^, . . . ,P.„i3j), (g,.,i3J, . . . ,g.„i3j)) = n[/*(l)]"- 

The isoperimetric inequality proved in [39] (Theorem 3.2) says that under certain conditions on ft and Ki, 



Dri(QK,,- ■ ■ , QkJ, (Pk,,. ■.,PkJ)< lllMl)]' 



/r. 



and the maximum is obtained when all Ki are centrally symmetric Euclidean balls. This inequality will be extended 
to a more general setting for the mixed /-divergence for measures. 

B. The f -dissimilarity for multiple convex bodies. 

The above connection between general mixed affine surface areas and the mixed /-divergence can be further 
extended to the f-dissimilarity for multiple convex bodies. Let f : R' ^ R be a convex function. We consider the 
measure space (5"^^, a). For a convex body K G /Co of class of C\, Pk is a probability measure associated with 
K. We denote by px the density function of Pk with respect to a. Likewise, for convex bodies 7^; G /Co, 1 < i < /, 
of class of C^, we let Pk^ be measures associated with Ki whose Radon-Nikodym derivatives with respect to the 
spherical measure a are pKi ■ Then the f-dissimilarity of Pxi with reference probability measure Pk is defined as 

Df{PK,,--- ,Pk,;Pk)= I i(^^,--- ,^PKda. 

Js^-i \PK PK J 

We will also use the notation Df{pKi,- ' ' iPKi',Pk) for Df{PKi,- ■ ■ ,Pki',Pk) and Df{p,q) for Df{P,Q). 
Aside from the general (mixed) affine surface areas, many other important objects in convex geometry are special 

cases of the /-dissimilarities. We now give another example. Let K = B2, and Pk = \q'b^\ - ^^^ -^1 ^ ^0 be of 

■2 

\dKi\ 



class of C^. Then, using the convex function f{x) = x, the surface area of dKi is 



^L-/^^(")'^" = ''^(^^'-^ 



Similarly, using the convex function f{x) = x~^ , a; > 0, one can also write |9A'i| as 



|9BJ| ' V ' \dB'i\ 

By Jensen's inequality (see also inequality ( IIV.8I )), one has 

n + l 

\dKi\ ( as{Ki)\ " ( as{Ki) 



\dB^i\- \n\B^\ ) \as{B^i), 

which compares the surface area and the affine surface area (e.g. ||6l, ifTTl . Il20ll . Il32ll ) 

as{K^)^ f [fKA^)]^da. 

Note that f{x) ~ x~!^ is strictly convex. Thus equality holds in the above inequality if and only if JkAu) = C, 
with C > a constant, which happens if and only if A'l is a Euclidean ball. 
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IV. Inequalities. 

The classical Alexandrov-Fenchel inequality for mixed volumes of convex bodies is a fundamental result in 
(convex) geometry. A general version of this inequality for mixed volumes of convex bodies A'l , . . . , A'„ in M" 
(see lH], Q, II3TI ) can be written as, for all integer m s.t. 1 < m < n 

m—l 
T[V{Ki, ■ ■ ■,Kn-m,Kn-i, ■ ' ; Kn-i) <V'^ {Ki, ■ ■ •, A„)- 

t% ' ■ ' 

m 

We refer to e.g. lISTI for the definition of the mixed volume V{Ki, ■ ■ ■ , Kn) of the bodies A'l, . . . , Kn- 

Alexandrov-Fenchel type inequality for the (mixed) affine surface areas can be found in |fT9l . ||201 . Il38l . ||39l . 
Now we prove an Alexandrov-Fenchel type inequality for the mixed /-divergence for measures. 

Following ifTSll . we say that two functions / and g are effectively proportional if there are constants a and b, not 
both zero, such that af = bg. Functions /i, . . . , /,„ are effectively proportional if every pair {fi, fj),l < i,j < m 
is effectively proportional. A null function is effectively proportional to any function. These notions will be used 
in the next theorems. 

For a measure space (X, ^) and probability densities pi and g^, 1 < i < n, we put 



50 (u) = n 

4 = 1 

and for j = 0, • • • , to — 1, 



n—ra 

P 



/.I -I* 



Jn-j [ I qn-j 

Hn-j 



(IV.6) 



(IV.7) 



9j+i{u) 
For a vector p, we denote by p "''^ the following vector 

P "■''' == (Pi,--- ,Pn^m,Pk,--- ,Pk), k > n-m. 

•• V ' 

m 

Theorem IV.l. Let (X, fi) be a finite measure space. For 1 < i < n, let Pi and Qi be probability measures on 
{X,fi) with density functions pi and qi respectively almost everywhere w.rt.fx. Let fi : (0,oo) — ^ M+, \ < i < n, 
be convex functions. Then, for 1 < m < n, 

n 

[Df{p,Q)r< n %-.,.(?"'". Q"'")- 

fc — n — m + l 

Equality holds if and only if one of the functions g^^ Qi, 1 < ^ ^ ?^^ is null or all are effectively proportional ji-a.e. 
If m = n, 

n 
i=l 

with equality if and only if one of the functions fj{ — ) Qj, < j < n, is null or all are effectively proportional 
ji-a.e. 

Remarks, (i) In particular, equality holds in Theorem II V. 1 1 if all (P^, Qi) coincide, and fi = Xif for some convex 
positive function / and Ai > 0, i = 1, 2, • ■ • , 71. 
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(ii) Theorem II V. 1 1 still holds true if the functions fi are concave. 

Proof of Theorem \IV.l\ We let g^ and gj+i, j = 0, ■ • ■ , m — 1 as in (IIV.6I) and dlV.Tb . By Holder's inequality 
(see ini) 



m— 1 

n [9o{u)gj+i{u)"^]~- dn 

3=0 

rn— i. / „ 

n ( / 9o{u)g]lj{u)dfi 



3=0 

m~ 1 

< 

3=0 



fc — n — Tn + l 

By e.g. lfT3l . equality holds in Holder's inequality, if and only if one of the functions g^^ gu 1 < « < m, is null 
or all are effectively proportional /i-a.e.. 

In particular, this is the case, if for all i = 1, • • • , n, (P^, Qi) — (P, Q) and fi — Xif for some convex function 
/ and X^>0. ■ 

Let / : (0, cxd) -^ R+ be a convex function. By Jensen's inequality, 

Df{P,Q) = j f(^ qdf, > f(J Pdl^ = /(I), (IV.8) 

for all pairs of probability measures (P, Q) on {X, ji) with nonzero density functions p and q respectively almost 
everywhere w.rt./i. When / is linear, equality holds trivially in (II V. 8) . When / is strictly convex, equality holds 
true if and only if p = q almost everywhere with respect to the measure /i. If / is a concave function, again by 
Jensen's inequality, 

Df{P,Q)^jj{^ qdfi < fUpd^^ = /(!)> aV.9) 

for all pairs of probability measures (P, Q). Again, when / is linear, equality holds trivially. When / is strictly 
concave, equality holds true if and only if p = q almost everywhere with respect to the measure fi. 

For the mixed /-divergence with concave functions, one has the following result. 

Theorem IV.2. Let {X, ii) be a finite measure space. For all 1 < i < n, let Pi and Qi be probability measures on 
X whose density functions pi and qi are nonzero almost everywhere w.rt.fi. Let fi : (0, cxd) -^ R+, 1 < i < n, be 
concave functions. Then 

n n 

[i^f(P,Q)]" < X{Du{P^,Qi) < l[.h{i). (IV.IO) 

i=l i=l 

If in addition, all fi are strictly concave, equality holds if and only if there is a probability density p such that 
almost everywhere with respect to the measure /i. 
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Proof: Theorem II V. 1 1 and the remark after imply that for all concave functions fi, 

n n 

where the second inequality follows from inequality ( |IV.9t and /^ > 0. 

Suppose now that for all i, p^ = qi = p, fi-a.e., where p is a fixed probability density. Then equality holds 
trivially in (lIV.lOl i. 

Conversely, suppose that equality holds in dlV. lOl l. Then, in particular, equality holds in Jensen's inequality which, 
as noted above, happens if and only if pi = qi for all i. Thus, 



D^{P, Q) = (J[[M1)VA 1^ ql^'' . . . q]J-dn. 



Note also that if all fi : (0, cxa) -^ R+ are strictly concave, /;(!) j^ for all 1 < i < n. Equality characterization 
in Holder's inequaUty implies that all qi are effectively proportional /x-a.e. As all qi are probability measures, they 
are all equal (almost everywhere w.rt./i) to a probability measure with density function (say) p. ■ 

Remark. If fi{t) = ait + hi are all linear and positive, then equality holds if and only if all pi, qi are equal (almost 
everywhere w.rt. jj) as convex combinations, i.e., if and only if for all i, j 

tti bi flj bj 



flj + bi a, + bi ttj + bj Gj + bj 

V. The i-TH MIXED /-divergence. 
Let {X, p) be a finite measure space. Throughout this section, we assume that the functions 

/i,/2:(0,oo)^{a;eM:a;>0}, 

are convex or concave, and that Pi, P2, Qi, Q2 are probability measures on X with density functions pi,P2, Qi, 92 
which are nonzero almost everywhere w.rt. the measure /^j. We also write 

/=(/l,/2), P=(Pl,P2), Q=(Ql,Q2). 

Definition V.l. Let i G K. The i-th mixed f -divergence for (/, P, Q), denoted by D AP^ Q; i), is defined as 



D^iP,Q;i)^J 



MfW 



/2(g)<Z2 



dp. (V.ll) 



Remarks. Note that the i-th mixed /-divergence is defined for any combination of convexity and concavity of /i 
and /2, namely, both /i and /2 concave, or both /i and /2 convex, or one is convex the other is concave. 
It is easily checked that 

If < J < n is an integer, then the triple (/i, Pi, Qi) appears i-times while the triple (/2, P2, Q2) appears {n — i) 
times in D AP,Q;i). 
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For 1 = 0, 



and for i = n. 



D^iP,Q;i)=Df,{P2,Q2), 



Dj-{P,Q-i)=Df,{PuQi). 

Another special case is when -P2 = Q2 = M almost everywhere and ji is also a probability measure. Then such 
an i-th mixed /-divergence, denoted by Z?((/i, Pi, Qi), i; 72), has the form 



Examples and Applications. 

(i) For /(i) = |t — 1|, we get the i-th mixed total variation 



X 



/.I^H. 



d/i. 






(ii) For /i(<) = /2(i) = [ilnt]-|_, we get the (modified) i-th mixed relative entropy or i-th mixed Kullback Leibler 
divergence 



Dkl{P,Q]i) = 



X 



1 I ^1 
Pi In — 

91 



1 I ^2 
P2 In — 

92 



d/i. 



(iii) For the convex or concave functions /„, (t) = f"^, 7 = 1, 2, we get the i-th mixed Hellinger integrals 



D 



(/«l,/c2) 



Ix 



{ Oi'y 1 — a^ 

(P2"92 ■ 



d[i. 



In particular, for a^ = a, for j = 1, 2, 



This integral can be used to define the i-th mixed a-Renyi divergence 

1 



DJP,Q-t] 



a — 1 



In 



DiUjAP'Q-^^) 



The case a^ = ^ for all i gives 



-0(Vt,Vt)(^'^;*) == / (Pi9i)'" (P292) "" c?/x, 



X 



the i-f/i mixed Bhattcharyya coefficient or i-f/z mixed Bhattcharyya distance of the pi and g^. 

(iv) Important applications are again in the theory of convex bodies. As in section 2, let Ki^ K2 G /Co be convex 

bodies with positive curvature function. For ^ = 1, 2, let 



PkM) = 



1 



n\Kt\hlXO 



QkM) 



lKM)hKM) 



n \Ki\ 



and define probability measures on S*" ^ by 

Pki^PKiO- and Qk, = qK,cr. 
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Let // : (0,00) ^ M, Z = 1,2, be positive convex functions. Then, we define the i-th mixed f -divergence for the 
convex bodies A'l and K2 by 

dA{Pk,,Pk,)AQk„Qk,);i) 



Kl\-^\K^\ \ fKM^ 

fK,hl+' i n\Ki\ 



h 



lAir^lA'al \ fK,hK2 



iK.hlX^ 



n\K-i 



da. 



This expression is closely related to the general i-th mixed L^ (or L,p) affine surface areas introduced in jW 
Similarly, 



5„-i 



n+l 



fKih^K^ 



1 



/2 



K? 



Ik2^ 



Ki 



n+l 
K2 



1 



\K°\-^K2\ / nlK^lh'l^ 



da, 



which is closely related to the general ith mixed L*. (or LV) affine surface areas introduced in 
The following result holds for all possible combinations of convexity and concavity of /i and /2. 



Proposition V.l. Let /, P, Q he as above. Ifj<i<kork<i<j, then 



DAP,Q-i)< 



Df{P,Q;j) 



Df{P,Q;k) 



Equality holds trivially if i — k or i = j. Otherwise, equality holds if and only if one of the functions fi 
i = 1,2, is null, or fi[ — ]qi and f2\ — )<l2 cire effectively proportional fi-a.e. In particular, this 
(Pi, Qi) = (P2, (52) and /i == //2 for some I > 0. 



Qt, 



holds if 



Proof: By formula (IV. 1 11 1, one has 

Df{P,Q;i) 



/.la,. 



/.i|i,. 



/.ij,« 



h[B.,^ 



d^ 






^^(fl^^ 



MglQ^ 



djj, 



Df{P,Q;j) 



Df{P,Q;k) 



where the last inequality follows from Holder's inequality and formula dV. lit . The equality characterization follows 
from the one in Holder inequality. In particular, if (Pi,Qi) = (P2,Q2), and /i = lf2 for some I > 0, equality 
holds. ■ 
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Corollary V.l. Let /i and /2 be positive, concave functions on (0, cxd). Then for all P, Q and for all < i < n, 

[i?/(P,g;*)]"<[/i(i)]M/2(i)]"-^ 

If in addition, /i and fi are strictly concave, equality holds if and only if pi = P2 = Qi = Q2 p-a.e. 



Proof: Let j = and fc = n in Proposition IV. 11 Then for all < i < n, 

where the last inequality follows from inequality (|IV.9t . 

To have equality, the above inequalities should be equalities. By Proposition IV. II one has then that fi [—) qi 
and f2 [ — ) 12 are effectively proportional /i-a.e. As both, /i and /2, are strictly concave, Jensen's inequality 
requires that pi ~ qi and p2 = (?2 /i-a.e. Therefore, equality holds if and only if /i(l)gi and ./2(1)92 are effectively 
proportional /i-a.e. As both, /i(l) and /2(1), are not zero, equality holds iff Pi = P2 = qi = 92 /i-a.e. ■ 

Remark. If /i(t) = ait + bi and /2(i) = a2t + &2 are both linear, equality holds in Corollary IV. 1 1 if and only if 
Pi, qi, i — 1,2, are equal as convex combinations, i.e., if and only if 



oi 6i a2 b; 



'2 



ai + bi ai + 61 02 + 02 02 + 02 

This proof can be used to establish the following result for Z3((/i, Pi, Qi), i; /2). 

Corollary V.2. Let {X, fi) be a probability space. Let /i be a positive concave function on (0, oo). Then for all 
Pi, Qi, for all (concave or convex) positive functions /2, and for all < i < n, 

[D{{fi,Pi,Qi),i;f2)r < [/i(l)]M/2(l)]"-^ 

If fi is strictly concave, equality holds if and only if Pi = Qi = fJ- When fi{t) = at + b is linear, equality holds 
if and only if api + bqi = a + b /i-a.e. 

Corollary V.3. Let /i be a positive convex function and /2 be a positive concave function on (0, cx)). Then, for all 
P, Q, and for all k > n, 

[^/(P,0;fc)]">[./i(ir[/2(i)]"-'=. 

If in addition, /i is strictly convex and /2 is strictly concave, equality holds if and only if pi = P2 = qi =92 p-a.e. 
Proof: On the right hand side of Proposition IV. 1 1 let i = n and j = 0. Let k > n. Then 

[i?/(Ao;fc)]" > [Df,{Pi,Qi)nDf,{P2,Q2)r-' 
> [h{i)nf2ii)r-''. 
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Here, the last inequality follows from inequalities ( |IV.8t . ( IIV.9I ) and k > n. To have equality, the above inequalities 
should be equaUties. By Proposition IV. II one has then that fi\ — ]qi and /2 ( — ) 92 are effectively proportional 
/^-a.e. As /i is strictly convex and /2 is strictly concave, Jensen's inequality implies that pi = qi and p2 = q-i 
fi-a.e. Therefore, as both /i(l) and /2(1) are not zero, equality holds if and only if pi = P2 = qi = qi /i-a.e. ■ 

Remark. If /i(t) = ait + bi and /2(i) = 02^ + 62 are both linear, equality holds in Corollary IV. 3 1 if and only if 
Pi,qi, i = 1,2, are equal /i-a.e. as convex combinations, i.e., if and only if 



oi 61 02 h 



fli + bi fli + 61 02 + 62 02 + 62 

This proof can be used to establish the following results for Z3((/i, Pi, Qi), k; f-i). 

Corollary V.4. Let {X,fi) be a probability space. Let /i be a positive convex function on (0, cxd). Then for all 
Pi, Qi, for all (positive concave or convex) functions J2, and for all k > n, 

[D{{fi,Pi,Qi),k;f2)r > [/i(l)]'=[/2(ir-^ 

If /i is strictly convex, equality holds if and only if Pi = Qi = P- When /i (t) = at + b is linear, equality holds if 
and only if api + bqi = a + b p-a.e. 

Corollary V.5. Let fi be a positive concave function and /2 be a positive convex function on (0, 00). Then for all 
P, Q, and for all fc < 0, 

[/?/(AQ;fc)]">[/i(ir[/2(i)]"-^ 

If in addition, /i is strictly concave and J2 is strictly convex, equality holds iff pi = P2 = 91 = 92 p-a.e. 

Proof: Let i = and j = n in Proposition IV. II Then 

[D^-{P,Q;k)y' > [Df,{Pi,Qi)f[Df,iP2,Q2T-'' 

> [./i(i)]'-[/2(ir-^ 

Here, the last inequality follows from inequalities (llV.Sb . ( |IV.9t , and fc < 0. 

To have equality, the above inequalities should be equalities. By Proposition lV.il one has then that /i ( — ) 9i and 
/2 ( — ) 92 are effectively proportional p,-a.e. As /i is strictly concave and /2 is strictly convex, Jensen's inequality 
requires that pi = 91 and p2 = 92. Therefore, equality holds if and only if fi{l)qi and /2(1)92 are effectively 
proportional /i-a.e. As both /i(l) and /2(1) are not zero, equality holds if and only if pi = P2 = 9i = 92 p-a.e. ■ 

This proof can be used to establish the following results for Z3((/i, Pi, Qi), fc; 72). 

Corollary V.6. Let /i be a concave function on (0, 00). Then for all Pi, Qi, for all (concave or convex) functions 
/2, and for all fc < 0, 

[D{{fi,Pi,Qi),k;f2)Y > [fiiltMir"". 
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If ,fi is strictly concave, equality holds if and only if Pi = Qi = fJ- When fi{t) = at + b is linear, equality holds 
if and only if api + bqi = a + b ii-a.e. 
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